Interacting with computing systems via natural interactions, such as one or more of voice recognition, gesture recognition, intent recognition, etc., enables natural user interface experiences. Such natural user interface experiences can be augmented when computing systems have information regarding semantic identifiers of the objects in their environment.